Master/Slave Speculative Parallelization with Distilled Programs

Speculative multithreading holds the potential to substantially improve the execution performance of sequential programs by leveraging the resources of multiple execution contexts (e.g., processors or threads). For unstructured non-numeric programs, the three key challenges of parallelization are (1) predicting the sequence of tasks (i.e., groups of instructions) that corresponds to the correct sequential execution of the program, (2) providing each task with the values it needs to execute (i.e., its live-in values), and (3) tolerating the communication latency between processors. To overcome these challenges, we propose a new execution model that differs from previous speculative multithreading models because of its master/slave nature. In this model, one execution context—the master—executes a speculative approximation of the original program—the distilled program—that allows it to anticipate future controland data-flow and explicitly orchestrate the parallel execution. Because the control flow in the two programs roughly corresponds, the master can accurately predict the sequence of tasks by mapping its program counter (PC) in the distilled program to a task start PC in the original program. Furthermore, speculative state (e.g., register and memory values) generated by the execution of the distilled program serves as predicted live-in values for the tasks. These predictions reduce the impact of communication latency on execution performance. We present an analytical model that shows that, if predictions made by the master are accurate, the execution performance closely tracks that of the distilled program. The distilled program can execute faster than the original program for two reasons: (1) it generates only a subset of the state generated by the original program, and (2) it need not perform the predictable portion of the computation, because the predictions will be verified by the parallelized execution of the original program. We perform an initial exploration of the potential of distilled programs, showing that dynamic instruction count can be reduced significantly with minimal impact on accuracy.